Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor! #153

Merged
merged 15 commits into from
Nov 5, 2024
Merged

Refactor! #153

merged 15 commits into from
Nov 5, 2024

Conversation

savq
Copy link
Collaborator

@savq savq commented Sep 21, 2024

Various changes based on the lezer-julia rewrite.


Notes for downstream:

The way type definitions are parsed now probably breaks existing locals.scm, and makes querying type name more complicated. The only upside is that patterns written for type_head apply to all abstract/primitive/struct definitions.

@savq savq force-pushed the lezer branch 4 times, most recently from 54d5fba to 803b97f Compare September 27, 2024 19:54
@savq savq marked this pull request as ready for review October 3, 2024 19:19
@savq savq force-pushed the lezer branch 2 times, most recently from 2ddd692 to 32c4606 Compare October 8, 2024 21:11
savq and others added 11 commits October 29, 2024 10:27
Not sure why inlining changes the behavior of the parser,
but `_top_level` has to be inlined, otherwise everything breaks.
- Remove `type_clause` rule
- Remove `type_parameter_list` aliases
- Update and organize tests for type definitions
- Add query patterns for `where_expression` and subtyping operators
- Simplify `signature` definition
- Remove `return_type` and `where_clause` hacks
- Arrow functions cannot contain open tuples, so they should only allow
  closed assignments (like `let_statement`).
- Rename `function_expression` to `arrow_function_expression`.
Parse the start of block_comment as a regular token.
Parse the start of a string as a regular token, and then parse the
content of the string using the external tokenizer. This requires rules
for all types of content (string/command, normal/raw, single/triple),
but it doesn't require using the external tokenizer to serialize string
delimiters.
- Remove unnecessary newlines from `_comprehension_clause` and
  `matrix_expression`.
- Update definition of `_terminator`.

Co-authored-by: Imran Khan <[email protected]>
- Rename `addDots` to `addDot`.
- Move `token` call to `addDot` definition.
- Add `public_statement`
- Split import rule into `import_statement` and `using_statement`
savq and others added 3 commits October 31, 2024 10:06
All types of content are aliased as content. Query patterns will usually
need to include the parent rule anyways.

Since the alias only includes content tokenized by the scanner, escape
sequences are _not_ included.

Co-authored-by: MichaelHatherly <[email protected]>
Inside raw strings, only escaped delimiters and \\ are considered escape
sequences. Other backslashes are treated as regular characters.
tree-sitter 0.24.3
@savq
Copy link
Collaborator Author

savq commented Nov 5, 2024

Fuzzer has a timeout error, but the tests are fine. We ball.

@savq savq merged commit 3f51e8e into tree-sitter:master Nov 5, 2024
3 of 4 checks passed
@clason
Copy link

clason commented Nov 6, 2024

The way type definitions are parsed now probably breaks existing locals.scm, and makes querying type name more complicated.

It breaks ALL THE QUERIES (in nvim-treesitter) ;) Please PR!

@savq
Copy link
Collaborator Author

savq commented Nov 6, 2024

Yes, this release includes breaking changes, detailed in the queries we track here. I mentioned locals.scm specifically because that's not included here.

I'm now only using the queries in this repo to ensure they work, and because I got tired of Neovim crashing and never pointing out the actual error in the queries :/

@clason
Copy link

clason commented Nov 6, 2024

I'm sorry to hear that (not least because that is an unfair complaint); it seems like this will mean that Julia will no longer be tier 1 parser (and eventually dropped) in Neovim.

fredrikekre added a commit to fredrikekre/nvim-treesitter-textobjects that referenced this pull request Nov 22, 2024
This patch adapts the julia queries to the breaking changes in
tree-sitter/tree-sitter-julia#153.

fixes nvim-treesitter#716
clason pushed a commit to nvim-treesitter/nvim-treesitter-textobjects that referenced this pull request Nov 22, 2024
This patch adapts the julia queries to the breaking changes in
tree-sitter/tree-sitter-julia#153.

fixes #716
fredrikekre added a commit to fredrikekre/nvim-treesitter that referenced this pull request Nov 22, 2024
This patch fixes the julia `(string_literal)` injection queries after
the breaking changes in
tree-sitter/tree-sitter-julia#153. The queries
are simplified by the fact that string content is now directly available
as a separate `(content)` child node.
clason pushed a commit to nvim-treesitter/nvim-treesitter that referenced this pull request Nov 22, 2024
This patch fixes the julia `(string_literal)` injection queries after
the breaking changes in
tree-sitter/tree-sitter-julia#153. The queries
are simplified by the fact that string content is now directly available
as a separate `(content)` child node.
fredrikekre added a commit to fredrikekre/nvim-treesitter that referenced this pull request Nov 22, 2024
This fixes the macro identifier highlight query after the breaking
changes in tree-sitter/tree-sitter-julia#153.
See also
tree-sitter/tree-sitter-julia@9f2fb3b.
clason pushed a commit to nvim-treesitter/nvim-treesitter that referenced this pull request Nov 22, 2024
This fixes the macro identifier highlight query after the breaking
changes in tree-sitter/tree-sitter-julia#153.
See also
tree-sitter/tree-sitter-julia@9f2fb3b.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants